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Abstract 

This paper seeks to outline methods underlying 
the development of the Xth Sense project, an 
ongoing research which investigates exploratory 
applications of biophysical sound design for 
musical performance and responsive milieux. 
Eirstly, an aesthetical study of body sounds, 
namely muscle sounds is illustrated. I describe the 
development of an audio synthesis model for 
muscle sounds' which offered a deeper 
understanding of the body sound matter and 
provided the ground for further experimentations 
in signal processing and composition. Then 
follows a description of the development and 
design of the Xth Sense, a wearable hardware 
sensor device for capturing biological body 
sounds; this was implemented in the realization of 
Music for Flesh I, a first attempt at musical 
performance. Next, the array of principles 
underpinning the application of muscle sounds to a 
musical performance is illustrated. Drawing from 
such principles, I eventually describe the methods 
by which useful features were extracted from the 
muscle sounds, and the mapping techniques used 
to deploy these features as control data for real 
time sound processing. 
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1 Introduction 

Biosensing musical technologies use biological 
signals of a human subject to control music. One 
of the earliest applications can be identified in 
Alvin Eucier's Music for Solo Performer (1965). 
Alpha waves generated when the performer enters 

* A digital muscle sounds generator. 


a peculiar mind state are transduced into electrical 
signals used to vibrate percussion instruments. 
Over the past thirty years biosensing technologies 
have been comprehensively studied [3, 8, 13, 14, 
15, 18, 22] and presently notable biophysical-only 
music performances^ are being implemented at 
SARC^ by a research group lead by the main 
contributor to the Bio Muse project"' Ben Knapp 
( 10 ). 

Whereas biological motion and movement and 
music are arising topics of interest in neuroscience 
research [5, 12, 21], the biologic body is being 
studied by music researchers as a mean to control 
virtual instruments. Although such approach has 
informed gestural control of music, I argue that it 
overlooks the expressive capabilities of biological 
sounds produced by the body. They are inaudible 
but may retain a meaningful vocabulary of 
intimate interactions with the musicians' actions. 

To what extent could biologic sounds be 
employed musically? In which ways could the 
performer's perceptual experience be affected? 
How could such experimental paradigm motivate 
an original perspective on musical performance? 

2 Aesthetic principles 

The long-term outcome of the research is the 
implementation of low cost, open source tools 
(software and hardware) capable of providing 
musicians, performers and dancers with a 
framework for biosensors-aided auditive design 
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Bio Muse Trio, GroundMe!. 

^ Queen's University, Sonic Art Research Center, Belfast, UK. 

^ A commercialized product exploiting electromyography and 
brainwaves analysis systems for musical applications. 



(BAAD)^ in a real time® environment; which 
framework will he re-distrihutahle, customizahle 
and easy to set up. However, given the substantial 
interdisciplinary quality of such project, its 
realization process needed to he fragmented into 
more specific and measurahle steps. 

The primary aim of the inquiry was to explore 
the musical and design capabilities of biological 
sounds of the body in a functional context - the 
production of Music for Flesh I a sonic solo 
performance for wearable biosensing device, 
which could demonstrate an experimental coupling 
between theatrical gesture and muscle sounds. In 
an attempt to inform the present state of 
augmented musical performance and embodied 
interaction, the characteristics of this pairing were 
identified in: the authenticity of the performer's 
somatic interaction, the natural responsiveness of 
the system and the expressive immediacy and 
transparency of the mapping of biological sound to 
the performer's kinetic behaviour. Such work 
required an interdisciplinary approach embracing 
biomedical computing studies, music technology 
and most importantly sound design. In fact, as I 
will demonstrate later in this text, the major 
research issue was not a technical implementation, 
but rather the definition of design paradigms by 
which the captured biological sounds could 
achieve a meaningful and detailed expressiveness. 

3 Methods: understanding and capturing 
muscle sounds 

The earliest approach to muscle sounds 
consisted of an analysis of the physical phenomena 
which makes muscle vibrate and sound. This study 
eventually developed in a sound synthesis model 
of muscle sounds. Although such model was not 
strictly related to the physical properties of muscle 
sounds, but rather to their aesthetic characteristics, 
it provided sonic samples which would satisfyingly 
resemble the original ones. Thereafter, the 
synthesised samples were used to explore design 
methodologies, while the sensor hardware 
implementation was still in progress. 


^ BAAD is a novel term used by the author to indicate a specific 
sound design practice which relies on the use of biological signals. 
Although in this context is not possible to further elaborate on this 
practice, its essential principles are defined in paragraph 4.1. 

^ Real time refers here to a computing system in which there exists 
no perceivable delay between performer's actions and sonic response. 


Following this initial study, the scope of the 
research consisted of two interrelated strands. The 
first concerned the design and implementation of a 
wearable biosensing hardware device for musical 
performance; the second included the development 
of a tracking system for a performer's somatic 
behaviour by means of muscle sounds features 
extraction and data mapping methods. 

The study of a synthesis model of muscle sounds 
is described in the next paragraph, whereas the 
research methods employed during the hardware 
and software design are discussed in the following 
paragraphs; however, being the focus of this paper 
on the research methodology, specific signal 
processing techniques and other technical 
information are not illustrated in detail, but they 
are fully referenced. 

3.1 An audio synthesis model of muscle 
sounds 

Muscles are formed by several layers of 
contractile filaments. Each of them can stretch and 
move past the other, vibrating at a very low 
frequency. However, audio recordings of muscle 
sounds show that their sonic response is not 
constant, instead it sounds more similar to a low 
and deep rumbling impulse. This might happen 
because each filament does not vibrate in unison 
with each other, but rather each one of them 
undergoes slightly different forces depending on 
its position and dimension, therefore filaments 
vibrate at different frequencies. Eventually each 
partial (defined here as the single frequency of a 
specific filament) is summed to the others living in 
the same muscle fibre, which in turn are summed 
to the muscle fibres living in the surrounding 
fascicle. 

Such phenomena creates a subtle, complex audio 
spectra which can be synthesised using discrete 
summation formula (DSE). This technique allows 
the synthesis of harmonic and in-harmonic, band- 
limited or unlimited spectra, and can be controlled 
by an index [7], which seemed to fit the 
requirement of such acoustic experiment. 

Being that the use of open source technologies is 
an integral part of the project, a Einux operating 
system was chosen as development environment. 
Muscle sound audio synthesis model was 
implemented using the open source framework 
known as Pure Data (Puckette 1996), a graphical 
programming language which offers a flexible and 



powerful architecture for real time sonic synthesis 
and processing. DSF was first used to generate the 
fundamental sidebands of the model; then the same 
formula was applied to a noise generator in order 
to add some light distortion to the model hy means 
of complex spectra formed hy small, slow noise 
hursts. Filter hanks were applied to each spectra in 
order to emphasise specific harmonics, thus 
refining the design. Eventually the two layers were 
summed, passed through a further filter hank and a 
tanh function’, which added a more natural 
characteristic to the resulting impulse. The model 
also included an automated random envelope 
generator used to constantly change the duration 
and intensity of individual impulses, thus better 
simulating a human muscle contraction. 

Model was then embedded in a parent patch® in 
order to evaluate the suitability of diverse signal 
processing techniques. Although testing showed 
interesting results with most of the applied 
processes, single-sideband pitch shifting (SSB 
modulation) proved to be the most meaningful 
method; in fact, being the muscle resonance 
frequency so low to not be immediately 
perceivable to human ear, namely between 5Hz 
and 40/45HZ, it would result difficult to produce 
heterogeneous sonic material to be used in a 
musical performance. SSB modulation [4] 
disclosed a new viewpoint on the further use of 
muscle sounds, allowing me to shift the initial 
spectrum of muscle fibre sound to a higher 
frequency range®; such method enriched the 
musical pitch range of muscles, prompting the 
composition of a more elaborate score. 

3.2 Xth Sense: first prototype sensor 
implementation 

Before undertaking the development of the Xth 
Sense sensor hardware, few crucial criteria were 
defined: 

• to develop a wearable, unobtrusive device, 
allowing a performer to freely move on 
stage; 


^ See TANH. Available at: 

http://idlastro.gsfc.nasa.gov/idl html help/TANH.html [Accessed 
January 12, 2011]. 

^ In this context the term 'patch' refers to a Pure Data-based 
application. 

^ It was interesting to note that pitch-shifted muscles sounds quite 
closely resemble a plucked chord. 


• to implement an extremely sensitive 
hardware device which could efficiently 
capture in real time and with very low 
latency diverse muscle sounds; 

• to make use of the most inexpensive 
hardware solutions, assuring a low 
implementation cost; 

• to implement the most accessible and 
straightforward production methodology in 
order to foster the future re-distribution 
and openness of the hardware. 

Study of the hardware sensor design began with a 
contextual review of biomedical engineering 
papers and publications focused on mechanical 
myography (MMG). The mechanical signal which 
can be observed from the surface of a muscle when 
it is contracted is called a MMG signal. At the 
onset of muscle contraction, significant changes in 
the muscle shape produce a large peak in the 
MMG. The oscillations of the muscles fibers at the 
resonance frequency of the muscle generate 
subsequent vibrations. The mechanomyogram is 
commonly known also as the phonomyogram, 
acoustic myogram, sound myogram or 
vibromyogram. 

Interestingly, MMG seems not to be a topic of 
interest in the study of gestural control of music 
and music technology; apparently many 
researchers in this fields focus their attention on 
electromyography (EMG), electroencephalography 
(EEG), or multidimensional control data which can 
be obtained through the use of wearable 
accelerometers, gyroscopes and other similar 
sensors. Notwithstanding the apparent lack of 
pertinent documentation in the studies of gestural 
control of music and music technologies, useful 
technical information regarding different MMG 
sensor designs were collected by reviewing the 
recent biomedical engineering literature. 

In fact, MMG is currently the subject of several 
investigations in this field as alternative control 
data for low cost, open source prosthetics research 
and for general biomedical applications [1, 6, 9, 
20]. Most notably the work of Jorge Silva'® at 
Prism Eab was essential to further advance the 
research; his MASc thesis extensively documents 
the design of the CMASP, a coupled microphone- 
accelerometer sensor pair (figure 1) and represents 

See: 

http://jsilva.koi'nodoopenlab.coi'n/index.php/Main/Research#toc6 






a comprehensive resource of information and 
technical insights on the use and analysis of MMG 
signals [19]. 

The device designed at Prism Lah is capable of 
capturing the audio signal of muscles sounds in 
real time. Muscle sonic resonance is transmitted to 
the skin, which in turn vibrates, exciting an air 
chamber. These vibrations are captured by an 
omnidirectional condenser microphone adequately 
shielded from noise and interferences by mean of a 
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Figure 1. CM ASP schematic 


silicon case. A printed circuit board (PCB) is used 
to couple the microphone with an accelerometer in 
order to filter out vibrations caused by global 
motion of the arm, and precisely identify muscle 
signals. Microphone sensitivity ranges from 20Hz 
up to 16kHz, thus it is capable of capturing a 
relevant part of the spectrum of muscles 
resonances". 

Although this design has been proved effectively 
functional through several academic reports, 
criteria of my investigation could have been 
satisfied with a less complex device. Supported by 
the research group at Dorkbot ALBA^^, I could 
develop a first, simpler MMG sensor: the circuit 
did not make use of a PCB and accelerometer, but 
deployed the same omnidirectional electret 
condenser microphone indicated by Silva 
(Panasonic WM-63PRT). This first prototype was 
successfully used to capture actual heart and 
forearm muscles sounds; earliest recordings and 
analysis of MMG signals were produced with the 
open source digital audio workstation Ardour2 and 
benchmark were set in order to evaluate the signal- 
to-noise ratio (SNR). 


It is interesting to observe that the remaining part of muscles 
sounds spectra seems to sit below 20Hz, thus pertaining to the realm 
of infra-sounds. Such characteristic is not being explored at the 
moment only due to technical constraints, although it suggests 
appealing prospects for a further research. 

Electronics open research group based in Edinburgh. 
http://dorkbot.noodlefactory.co.uk/wiki 


In spite of the positive results obtained with the 
first prototype, the microphone shielding required 
further trials. The importance of the shield was 
manifold; an optimal shield had to fit specific 
requirements: to bypass the 60Hz electrical 
interference which can be heard when alternating 
electric current distribute itself within the skin 
after a direct contact with the microphone metal 
case; to narrow the sensitive area of the 
microphone, filtering out external noises; to keep 
the microphone static, avoiding external air 
pressure that will affect the signal; to provide a 
suitable air chamber for the microphone, in order 
to amplify sonic vibrations of the muscles, and 
facilitating capture of deeper muscle contractions. 

First, the microphone was insulated by mean of 
a polyurethane shield, but due to the strong 
malleability of this material, its initial shape tended 
to flex easily. Eventually, the sensor was insulated 
in a common silicon case that satisfied the 
requirements and further enhanced the SNR. Once 
the early prototype had reached a good degree of 
efficiency and reliability, the circuit was embedded 
in a portable plastic box (3.15 x 1.57 x 0.67) along 
with an audio output (14 mono chassis jack socket) 
and a cell holder for a 3V coin lithium battery. 



Figure 2. Xth Sense wearable MMG sensor prototype 


Shielded microphone was embedded in a Velcro 
bracelet and needed wiring cables were connected 
to the circuit box (figure 2). 

4 Performance testing: mapping and design 
definitions 

At this stage of the project the understanding and 
creation of mapping and design paradigms for 
muscles sounds was the major goal. The main 














principles and some technical implementations are 
illustrated in the next paragraphs. 

4.1 Sound performance and design principles 

Major aim of the design of the MMG audio 
signals was to avoid a perception of the sound 
being dissociated from the performer's gesture. 
The dissociation I point at does not only refer to 
the visual feedhack of performer's actions being 
disjointed from the sonic experience, but it also, 
and most importantly, concerns a metaphorical 
level affecting the listener's interpretation of the 
sounds generated by the performer's somatic 
behavior [2]. In this project the use of muscle 
sounds had to be clearly motivated in order to 
inform classical approaches to gestural control of 
music. Therefore, chosen sound processing and 
data mapping techniques were evaluated according 
to their capability of enhancing the metaphorical 
interpretation of performer's physiological and 
spatial behaviour. 

In this perspective, the essential principles of 
BA AD in a performing environment were defined 
as follow: 

• to make use of biological sounds as major 
sonic source and control data; 

• to exclude the direct interaction of the 
performer with a computer and to conceal 
the latter from the view of the public; 

• to demonstrate a distinct, natural and non¬ 
linear interaction between kinetic energy 
and sonic outcome which could be 
instinctively controlled by the performer; 

• to provide a rich, specific and 
unconventional vocabulary of 
gesture/sound definitions which can be 
unambiguously interpreted by the 
audience; 

• to allow the performer to flexibly execute 
the composition, or even improvise a new 
one with the same sonic vocabulary; 

• to make both performer and public 
perceive the former's body as a musical 
instrument and its kinetic energy as an 
exclusive sound generating force. 

4.2 MMG features extraction 


evaluation and recognition of gestures, however 
muscle sounds resonance frequency is not affected 
by any external agent and its pitch seems not to 
change significantly with different movements 
[17]. Whereas muscles sounds are mostly short, 
discrete events with no meaningful pitch change 
information, the most interesting and unique aspect 
of their acoustic composition is their extremely 
rich and fast dynamic; therefore, extraction of 
useful data can be achieved by RMS amplitude 
analysis and tracking, contractions onset and 
gesture pattern recognition. In fact, each human 
muscle exerts a different amount of kinetic energy 
when contracting and a computing system can be 
trained in order to measure and recognize different 
levels of force, i.e. different gestures. Feature 
extraction enabled the performer to calibrate 
software parameters according to the different 
intensity of the contractions of each finger or the 
wrist and provided 8 variables: 6 discrete events, 1 
continuous moving event and 1 continuous 
exponential event. First, sensor was subjected to a 
series of movements and contractions with 
different intensity to identify a sensitivity range; 
this was measured between 57.79 dB (weakest 
contraction) and 89.04 dB (strongest contraction). 
The force threshold of each finger discrete 
contraction was set by normalizing and measuring 
the individual maximum force exertion level; 
although some minor issues arisen from the 
resemblance between the force amplitude exerted 
by the minimus (little finger) and the thumb still 
need to be solved, this method allowed the 
determination of 6 independent binary trigger 
control messages (fingers and wrist contractions). 

Secondly, by measuring the continuous 
amplitude average of the overall contractions, it 
was possible to extract the running maximum 
amplitude of performer's gestures; in order to 
correct the jitter of this data, which otherwise 
could not have been usefully deployed, value was 
extracted every 2 seconds, then interpolated with 
the prior one to generate a continuous event and 
eventually normalized to MIDI range. Lastly, a 
basic equation of single exponential smoothing 
(SES) was applied to the moving global RMS 
amplitude in order to forecast a less sensitive 
continuous control value [16]. 


Since the project dealt with sound data, a pitch 
tracking system would have possibly been a 
straightforward solution for an automated 



4.3 Mapping kinetic energy to control data 

A first mapping model deployed the 6 triggers 
previously described as control messages. These 
were used to enable the performer to control the 
real time SSB modulation algorhythm by choosing 
a specific frequency among six different preset 
frequencies; the performer could select which 
target frequency to apply according to the 
contracted finger; therefore, the voluntary 
contraction of a specific finger would enable the 
performer to “play” a certain note. 

A one-to-many mapping model, instead, used 
the continuous values obtained through the RMS 
analysis to control several processing parameters 
within five digital signal processing (DSP) chains 
simultaneously. Being that this paper does not 
offer enough room to fully describe the whole DSP 
system which was eventually implemented, I will 
concentrate on one example chain which can 
provide a relevant insight on the chosen mapping 
methodology; namely, this DSP chain included a 
SSB modulation algorhythm, a lofi distortion 
module, a stereo reverb, and a band-pass filter. 

The SSB algorhythm was employed to increase 
the original pitch of the raw muscle sounds by 
20Hz, thus making it more easily audible. 
Following an aesthetical choice, the amount of 
distorsion over the source audio signal was subtle 
and static, thus adding a light granulation to the 
body of the sound; therefore, the moving global 
RMS amplitude was mapped to the reverb decay 
time and to the moving frequency and Quality 
factor'^ (Q) of the band-pass filter. 

The most interesting performance feature of 
such mapping model consisted of the possibility to 
control a multi-layered processing of the MMG 
audio signal by exerting different amounts of 
kinetic energy. Stronger and wider gestures would 
generate sharp, higher resonating frequencies 
coupled with a very short reverb time, whereas 
weaker and more confined gestures would produce 
gentle, lower resonances with longer reverb time. 

Such direct interaction among the perceived 
force and spatiality of the gesture and the moving 
form and color of the sonic outcome happened 
with very low latency, and seemed to suggest 
promising further applications in a more complex 
DSP system. 


The Xth Sense framework was tested live during 
a first public performance of Music for Flesh I 
(figure 3) at the University of Edinburgh 
(December 2010). Although the system was still 
in development, it proved reliable and efficient. 
Audience feedback was positive, and apparently 
what most appealed some listeners was an 
authentic, neat and natural responsiveness of the 
system along with a suggestive and unconventional 
coupling of sound and gestures. 



Figure 3. Music for Flesh 1 first public performance, 2010 


5 Conclusions 

Results reported in this paper appear to disclose 
promising prospects of an experimental paradigm 
for musical performance based on MMG. The 
development of the Xth Sense and the composition 
and public performance of Music for Flesh I can 
possibly demonstrate an uncharted potential of 
biological sounds of the human body, specifically 
muscle sounds, in a musical performance. 

Notwithstanding the seeming rarity of interest of 
the relevant academic community towards the 
study and the use of these sounds, the experiment 
described here shows that muscle sounds could 
retain a relevant potential for an exploration of 
meaningful and unconventional sound-gesture 
metaphors. Besides, if compared to EMG and EEG 
sensing devices, the use of MMG sensors could 
depict a new prospect for a simpler 
implementation of unobtrusive and low-cost 
biosensing technologies for biophysical generation 
and control of music. 

Whereas the development of the sensor 
hardware device did not present complex issues. 
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Narrowness of the filter. 



several improvements to the tracking and mapping 
techniques can lead to a further enhancement of 
the expressive vocabulary of sound-gestures. In an 
attempt to enrich the performer's musical control 
over a longer period of time, hereafter priority will 
he given to the extraction of other useful features, 
to the development of a gesture pattern recognition 
system and to the implementation of polyphony, 
using two sensors simultaneously. 
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